home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
ftp.cs.arizona.edu
/
ftp.cs.arizona.edu.tar
/
ftp.cs.arizona.edu
/
icon
/
newsgrp
/
group00a.txt
/
000081_icon-group-sender _Thu Apr 27 12:31:53 2000.msg
< prev
next >
Wrap
Internet Message Format
|
2001-01-03
|
4KB
Return-Path: <icon-group-sender>
Received: (from root@localhost)
by baskerville.CS.Arizona.EDU (8.9.1a/8.9.1) id MAA24045
for icon-group-addresses; Thu, 27 Apr 2000 12:31:32 -0700 (MST)
Message-Id: <200004271931.MAA24045@baskerville.CS.Arizona.EDU>
From: "Frank J. Lhota" <NOSPAM.Frank.Lhota@lexma.meitech.com>
X-Newsgroups: comp.lang.icon
Subject: Re: Is Anyone Working On A Unicode Version Of Icon?
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 5.00.2919.6600
X-Mimeole: Produced By Microsoft MimeOLE V5.00.2919.6600
Date: Thu, 27 Apr 2000 12:35:56 -0000
X-Trace: client 956853371 38.163.203.81 (Thu, 27 Apr 2000 12:36:11 EDT)
To: icon-group@optima.CS.Arizona.EDU
Errors-To: icon-group-errors@optima.CS.Arizona.EDU
Status: RO
"Gregg Townsend" <gmt@CS.Arizona.EDU> wrote in message
news:8e75f6$5pr@baskerville.CS.Arizona.EDU...
> A Unicode version of Jcon (the Java implementation of Icon) would not
> be a huge task. The hardest problem would be deciding language issues,
> notably the definitions of the predefined csets.
Actually, even for the eight bit version of Icon, the definitions of
&letters, &lcase, and &ucase deserve a second look. For international use,
char( 16rE0 to 16rFE )
often represent lower case letters (such as "a" with an umlaut), and
char( 16rC0 to 16rDE )
often represent upper case letters, but none of these characters do not show
up in &letter, &lcase or &ucase. These predefined csets restrict their
attention exclusively to the &ascii characters.
Granted, this is due to the strange evolution of 8 bit character coding.
Standards for how to interpret characters with the high bit on were slow to
evolve. Some ASCII platforms had quite novel interpretations of these
characters. MS-DOS machines, for example, had the OEM character set, which
used most of the upper half of &cset for math symbols and symbols useful for
drawing boxes. Amidst this anarchy, it would be foolhardy for Icon to
include any characters outside of &ascii in &letters, &lcase, or &ucase.
We may, unfortunately, be at a point where this is too late to change the
predefined csets. There are probably Icon programs out there that depend on
*&letters = 52, *&lcase = *&ucase = 26, &letters ++ &ascii === &ascii, etc.,
and hence changing the csets would cause them to break. The right fix might
be to add new prefined csets, so that new code could be properly
internationalized.
> You'd also want a
> different internal representation for csets.
Currently, a cset is basically stored as a packed array of 256 bits (one bit
for each 8-bit character). For 32 bit platforms, this requires just 8 words.
If we were to use the packed array of bits approach for wide characters, a
cset would be represented by 2048 words -- Yikes! Clearly, we would need to
use a different internal representation.
I would suggest representing a cset as an ordered list of character
intervals. For example, the cset of characters that can be used in an Icon
identifier would be represented as this sequence of intervals:
[ "0" to "9", "A" to "Z", "_" to "_", "a" to "z" ]
I will grant that, in the worst case, this representation method can take up
a lot of storage. A cset of all wide characters c with the lowest order bit
on would require 32,768 intervals. Most csets that actually occur in
applications, however, could be represented by a handful of intervals.
I must confess that I am just starting to learn Java, so it will be awhile
before I can try this on my own :(